Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources

Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The objective of this work is to estimate the current state of the surgical task based on the actions performed or events occurred as the task progresses. We propose Fusion-KVE, a unified surgical state estimation model that incorporates multiple data sources including the Kinematics, Vision, and system Events. Additionally, we examine the strengths and weaknesses of different state estimation models in segmenting states with different representative features or levels of granularity. We evaluate our model on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), as well as a more complex dataset involving robotic intra-operative ultrasound (RIOUS) imaging, created using the da Vinci® Xi surgical system. Our model achieves a superior frame-wise state estimation accuracy up to 89.4%, which improves the state-of-the-art surgical state estimation models in both JIGSAWS suturing dataset and our RIOUS dataset

From principles to action: Applying the National Research Council's principles for effective decision support to the Federal Emergency Management Agency's watch office

AbstractThe National Research Council (NRC) proposed six principles for effective decision support in its 2009 report Informing Decisions in a Changing Climate. We structured a collaborative project between the Federal Emergency Management Agency Region R9 (FEMA R9), the Western Region Headquarters of the National Weather Service (WR-NWS), and the Climate Assessment of the Southwest (CLIMAS) at the University of Arizona around the application of the NRC principles. The goal of the project was to provide FEMA R9's Watch Office with climate information scaled to their temporal and spatial interests to aid them in assessing the potential risk of flood disasters. We found that we needed specific strategies and activities in order to apply the principles effectively. By using a set of established collaborative research approaches we were better able to assess FEMA R9's information needs and WR-NWS's capacity to meet those needs. Despite our diligent planning of engagement strategies, we still encountered some barriers to transitioning our decision support tool from research to operations. This paper describes our methods for planning and executing a three-party collaborative effort to provide climate services, the decision support tool developed through this process, and the lessons we will take from this deliberate collaborative process to our future work and implications of the NRC principles for the broader field of climate services

The Far-infrared Continuum of Quasars

ISO provides a key new far-infrared window through which to observe the multi-wavelength spectral energy distributions (SEDs) of quasars and active galactic nuclei (AGN). It allows us, for the first time, to observe a substantial fraction of the quasar population in the far-IR, and to obtain simultaneous, multi-wavelength observations from 5--200 microns. With these data we can study the behavior of the IR continuum in comparison with expectations from competing thermal and non-thermal models. A key to determining which mechanism dominates, is the measurement of the peak wavelength of the emission and the shape of the far-IR--mm turnover. Turnovers which are steeper than frequency^2.5 indicate thermal dust emission in the far-IR. Preliminary results from our ISO data show broad, fairly smooth, IR continuum emission with far-IR turnovers generally too steep to be explained by non-thermal synchrotron emission. Assuming thermal emission throughout leads to a wide inferred temperature range of 50-1000 K. The hotter material, often called the AGN component, probably originates in dust close to and heated by the central source, e.g. the ubiquitous molecular torus. The cooler emission is too strong to be due purely to cool, host galaxy dust, and so indicates either the presence of a starburst in addition to the AGN or AGN-heated dust covering a wider range of temperatures than present in the standard, optically thick torus models.Comment: 4 pages, to be published in the proceedings of "The Universe as Seen by ISO," ed. M. Kessler. This and related papers can be found at http://hea-www.harvard.edu/~ehooper/ISOkp/ISOkp.htm

Directed Acyclic Graph Continuous Max-Flow Image Segmentation for Unconstrained Label Orderings

Label ordering, the specification of subset–superset relationships for segmentation labels, has been of increasing interest in image segmentation as they allow for complex regions to be represented as a collection of simple parts. Recent advances in continuous max-flow segmentation have widely expanded the possible label orderings from binary background/foreground problems to extendable frameworks in which the label ordering can be specified. This article presents Directed Acyclic Graph Max-Flow image segmentation which is flexible enough to incorporate any label ordering without constraints. This framework uses augmented Lagrangian multipliers and primal–dual optimization to develop a highly parallelized solver implemented using GPGPU. This framework is validated on synthetic, natural, and medical images illustrating its general applicability

Infrared Properties of High Redshift and X-ray Selected AGN Samples

The NASA/ISO Key Project on active galactic nuclei (AGN) seeks to better understand the broad-band spectral energy distributions (SEDs) of these sources from radio to X-rays, with particular emphasis on infrared properties. The ISO sample includes a wide variety of AGN types and spans a large redshift range. Two subsamples are considered herein: 8 high-redshift (1 < z < 4.7) quasars; and 22 hard X-ray selected sources. The X-ray selected AGN show a wide range of IR continuum shapes, extending to cooler colors than the optical/radio sample of Elvis et al. (1994). Where a far-IR turnover is clearly observed, the slopes are < 2.5 in all but one case so that non-thermal emission remains a possibility. The highest redshift quasars show extremely strong, hot IR continua requiring ~ 100 solar masses of 500 - 1000 Kelvin dust with ~ 100 times weaker optical emission. Possible explanations for these unusual properties include: reflection of the optical light from material above/below a torus; strong obscuration of the optical continuum; or an intrinsic deficit of optical emission.Comment: 8 pages, 3 figures (2 color), to be published in the Springer Lecture Notes of Physics Series as part of the proceedings for "ISO Surveys of a Dusty Universe," a workshop held at Ringberg Castle, Germany, November 8 - 12, 1999. Requires latex style files for this series: cl2emult.cls, cropmark.sty, lnp.sty, sprmindx.sty, subeqnar.sty (included with submission

Temporal Segmentation of Surgical Sub-tasks through Deep Learning with Multiple Data Sources

Many tasks in robot-assisted surgeries (RAS) can be represented by finite-state machines (FSMs), where each state represents either an action (such as picking up a needle) or an observation (such as bleeding). A crucial step towards the automation of such surgical tasks is the temporal perception of the current surgical scene, which requires a real-time estimation of the states in the FSMs. The objective of this work is to estimate the current state of the surgical task based on the actions performed or events occurred as the task progresses. We propose Fusion-KVE, a unified surgical state estimation model that incorporates multiple data sources including the Kinematics, Vision, and system Events. Additionally, we examine the strengths and weaknesses of different state estimation models in segmenting states with different representative features or levels of granularity. We evaluate our model on the JHU-ISI Gesture and Skill Assessment Working Set (JIGSAWS), as well as a more complex dataset involving robotic intra-operative ultrasound (RIOUS) imaging, created using the da Vinci® Xi surgical system. Our model achieves a superior frame-wise state estimation accuracy up to 89.4%, which improves the state-of-the-art surgical state estimation models in both JIGSAWS suturing dataset and our RIOUS dataset

Free-breathing Pulmonary MR Imaging to Quantify Regional Ventilation

Purpose: To measure regional specific ventilation with free-breathing hydrogen 1 (1H) magnetic resonance (MR) imaging without exogenous contrast material and to investigate correlations with hyperpolarized helium 3 (3He) MR imaging and pulmonary function test measurements in healthy volunteers and patients with asthma. Materials and Methods: Subjects underwent free-breathing 1H and static breath-hold hyperpolarized 3He MR imaging as well as spirometry and plethysmography; participants were consecutively recruited between January and June 2017. Free-breathing 1H MR imaging was performed with an optimized balanced steady-state free-precession sequence; images were retrospectively grouped into tidal inspiration or tidal expiration volumes with exponentially weighted phase interpolation. MR imaging volumes were coregistered by using optical flow deformable registration to generate 1H MR imaging-derived specific ventilation maps. Hyperpolarized 3He MR imaging- and 1H MR imaging-derived specific ventilation maps were coregistered to quantify regional specific ventilation within hyperpolarized 3He MR imaging ventilation masks. Differences between groups were determined with the Mann-Whitney test and relationships were determined with Spearman (ρ) correlation coefficients. Statistical analyses were performed with software. Results: Thirty subjects (median age: 50 years; interquartile range [IQR]: 30 years), including 23 with asthma and seven healthy volunteers, were evaluated. Both 1H MR imaging-derived specific ventilation and hyperpolarized 3He MR imaging-derived ventilation percentage were significantly greater in healthy volunteers than in patients with asthma (specific ventilation: 0.14 [IQR: 0.05] vs 0.08 [IQR: 0.06], respectively, P \u3c .0001; ventilation percentage: 99% [IQR: 1%] vs 94% [IQR: 5%], P \u3c .0001). For all subjects, 1H MR imaging-derived specific ventilation correlated with plethysmography-derived specific ventilation (ρ = 0.54, P = .002) and hyperpolarized 3He MR imaging-derived ventilation percentage (ρ = 0.67, P \u3c .0001) as well as with forced expiratory volume in 1 second (FEV1) (ρ = 0.65, P = .0001), ratio of FEV1 to forced vital capacity (ρ = 0.75, P \u3c .0001), ratio of residual volume to total lung capacity (ρ = -0.68, P \u3c .0001), and airway resistance (ρ = -0.51, P = .004). 1H MR imaging-derived specific ventilation was significantly greater in the gravitational-dependent versus nondependent lung in healthy subjects (P = .02) but not in patients with asthma (P = .1). In patients with asthma, coregistered 1H MR imaging specific ventilation and hyperpolarized 3He MR imaging maps showed that specific ventilation was diminished in corresponding 3He MR imaging ventilation defects (0.05 ± 0.04) compared with well-ventilated regions (0.09 ± 0.05) (P \u3c .0001). Conclusion: 1H MR imaging-derived specific ventilation correlated with plethysmography-derived specific ventilation and ventilation defects seen by using hyperpolarized 3He MR imaging. © RSNA, 2018 Online supplemental material is available for this article

Understanding the effects of nutrition and post-exercise nutrition on skeletal muscle protein turnover: insights from stable isotope studies

Skeletal muscle is the largest organ of the human body and plays a pivotal role in whole-body homeostasis through the maintenance of physical and metabolic health. Establishing strategies aimed at increasing the amount, and minimising loss, of muscle mass are of upmost importance. Muscle mass is primarily dictated by the meal-to-meal fluctuations in muscle protein synthesis (MPS) and muscle protein breakdown (MPB), each of which can be quantified through the use of stable isotopically labelled tracers. Importantly, both MPS and MPB can be influenced by external factors such as nutritional manipulation, specifically protein ingestion, and changes in loading via exercise. To date, research involving stable isotopic tracers has focused on determining the optimal dose, timing surrounding bouts of exercise, distribution and composition of protein to maximally stimulate MPS and inhibit MPB, both at rest and following exercise. In this review we focus on the use of these stable isotopically-labeled tracers to unravel the intricacies of skeletal muscle protein turnover in response to specific nutritional interventions

An Unusual Transmission Spectrum for the Sub-Saturn KELT-11b Suggestive of a Sub-Solar Water Abundance

We present an optical-to-infrared transmission spectrum of the inflated sub-Saturn KELT-11b measured with the Transiting Exoplanet Survey Satellite (TESS), the Hubble Space Telescope (HST) Wide Field Camera 3 G141 spectroscopic grism, and the Spitzer Space Telescope (Spitzer) at 3.6 $\mu$m, in addition to a Spitzer 4.5 $\mu$m secondary eclipse. The precise HST transmission spectrum notably reveals a low-amplitude water feature with an unusual shape. Based on free retrieval analyses with varying molecular abundances, we find strong evidence for water absorption. Depending on model assumptions, we also find tentative evidence for other absorbers (HCN, TiO, and AlO). The retrieved water abundance is generally $\lesssim 0.1\times$ solar (0.001--0.7$\times$ solar over a range of model assumptions), several orders of magnitude lower than expected from planet formation models based on the solar system metallicity trend. We also consider chemical equilibrium and self-consistent 1D radiative-convective equilibrium model fits and find they too prefer low metallicities ($[M/H] \lesssim -2$, consistent with the free retrieval results). However, all the retrievals should be interpreted with some caution since they either require additional absorbers that are far out of chemical equilibrium to explain the shape of the spectrum or are simply poor fits to the data. Finally, we find the Spitzer secondary eclipse is indicative of full heat redistribution from KELT-11b's dayside to nightside, assuming a clear dayside. These potentially unusual results for KELT-11b's composition are suggestive of new challenges on the horizon for atmosphere and formation models in the face of increasingly precise measurements of exoplanet spectra.Comment: Accepted to The Astronomical Journal. 31 pages, 20 figures, 7 table